Feature Selection

Data dimensionality is growing exponentially, which poses challenges to the vast majority of existing mining and learning algorithms, such as the curse of dimensionality, large storage requirement, and high computational cost. Feature selection has been proven to be an effective and efficient way to prepare high dimensional data for data mining and machine learning. The recent emergence of novel techniques and new types of data and features not only advances existing feature selection research but also makes feature selection evolve more rapidly, becoming applicable to a broader range of applications. In this article, we aim to provide a basic introduction to feature selection including basic concepts, classifications of existing systems, recent development, and applications. Synonyms: Feature subset selection, Feature weighting, Attributes selection Definition (or Synopsis): Feature selection, as a dimensionality reduction technique, aims to choosing a small subset of the relevant features from the original ones by removing irrelevant, redundant or noisy features. Feature selection usually leads to better learning performance, i.e., higher learning accuracy, lower computational cost, and better model interpretability. Generally speaking, irrelevant features are features that cannot discriminate samples from different classes(supervised) or clusters(unsupervised). Removing irrelevant features will not affect learning performance. In fact, removal of irrelevant features may help learn a better model, as irrelevant features may confuse the learning system and cause memory and computation inefficiency. For example, in figure 1(a), f1 is a relevant feature because f1 can discriminate class1 and class2. In figure 1(b), f2 is a redundant feature because f2 cannot distinguish points from class1 and class2. Removal of f2 doesn’t affect the ability of f1 to distinguish samples from class1 and class2.